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The American Industry Project was initiated to develop and field jest a 
secondary school curriculum which had as its central purpose the study of industry. 
An evaluation system was adopted to provide data for curriculum design dxecisions 
and to measure program effects. Three courses were developed to .^ trod uce and 
provide experience in utilizing industrial concepts, involve studen s i 
activities and events in industry, and provide for individual study and proWem solving 
experience. The evaluation domains of ingredients, processes, and P r °®j c ji 
identified. Major findings related to student outcomes were. (1) 36-week courses 
produced more significant results than shorter courses. (2) Response patterns on 
attitude measures favored the American Industry students whe 
differences were obtained. (3) Attitudes toward industry and occupational behaviors 
were influenced most strongly. (4) The study of American Industry increacc^ 
perspectives of jobs available and their interest in seek ng employment one of t^ 
and (5) 75 percent of the students in both. the control and American Industry group 
were aware of the changing nature of (obs in industry. (DM) 
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THE EVALUATION SYSTEM FOR THE AMERICAN INDUSTRY SECONDARY SCHOOOL COURSES 



Introduction 

Throughout the last three years tho American Industry Project has been 
.developing and field- testing a secondary schoo3. curriculum which has as its . 

. central purpose the study of industry. An evaluation system was designed and 
refined in concert with the curriculum development activities to provide data 
. for the various curriculum design decisions and to measure the effects of tho 
study of American Industry. This. evaluation system is described in later 
sections of -this paper. . The following: section gives a brief overview of the 
American Industry rationale for those i-rho have not been exposed to it before* 
If the reader has already heard or read this material* he may desire to page 
directly to the section entitled "Evaluation of American Industry Courses." 

• This- section gives the rationale for the evaluation system developed for the 
Project and outlines the types .of- data collected. The concluding part of 
this paper presents' several studies based on- selected portions of the infor- 
mation 'gathered. • ‘ 7 .• • • * 

. * , t • 

; - Rationale for the Study of American Industry 

• During the 1962-63 school year, several -professors at Stout State 
University initiated a series of meetings to discuss their concerns related 
to the content and scope of the existing industrial arts curriculum. Out 
of these discussions grew the rationale for the study of American Industiy . 
Embodied in this rationale’ wore four basic tenets related, to tho source and 
nature of content to be studied. 

. . -The first tenet identified Industry as the source of. content. Industry 
was subsequently defined as "an institution in our society which, intending . 
to make a monetary profit, applies knowledge and utilizes natural and human 
resources to produce goods or services- to meet the needs of man." Obviously 
this definition encompasses a wide variety of business enterprises. To i% 
study each one would bo impossible. However, the discussion group noted a 
number of similarities among all enterprises. For example, merchants, 
automotive dealers, manufacturers, contractors, service station owners, and 
farmers have to apply management, techniques in -their businesses. 

.‘-The trend to the study of concepts in other curriculum areas reinforced 
the conclusion that a’ diverse set of enterprises could be reduced to a set 
of meaningful concepts. In addition, Bruner's work (.i 960 ) indicated that a 
discipline has an underlying structure which interrelates the basic ideas 
found in the discipline and lends considerable explanatory power to its 
postulates, concepts, and principles. On this basis, the decision was made 
to analyze a variety of industries, to isolate the concepts common to them, 
and to identify the relationships among tho concepts. After considerable 



^When used in this manner, industry rofors to a group of enterprises 
or businesses engaged in producing or servicing tho same typo of product such 
as the auto industry or the TV repair industry. American Industry 'ref ers to 
the institution identified in the Project’s definition of Industry and includes 
all industries that exist to make a monotary profit. •. *’• 
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study, several hundred contacts with industry and labor loaders, 
and considerable discussion on the part of tho American Industry 
staff, tho concepts and structure depicted in Figure 1 wore 
delineated. Tho concepts on the inner ball are tho ones Industry 
uses directly to accomplish its goals. The concepts listed in 
tho^ outer tfing identify the' environment in which Industry operates. 
An interaction relationship exists between tho concepts on the 
ball and those in the ring. 

Tenets two and three of tho four mentioned previously are 
related to this structure: (2) the study of American Industry will 

be concerned Tilth the . structure of the knowledges of Industry, and 
( 3 ) this ’study will, also concentrate on the concepts of Industry, 
The remaining tenet stresses the inclusion of problem solving 
experiences in the American Industry curriculum. ^ 

Three secondary school courses have been designed on the basis 
of this rationale and the structure. Tho first course, Level I, 
introduces the student to the basic concepts- of industry listed in 
Figure 1 and gives him experience using these concepts to solve 
■■industrial problems. In the second course, Level II j .the student 
goes’’ into greater depth in each of the concept areas and becomes . 
more involved with the actual flow of activities and events in an 
industry. At this level, ho also encounters more sophisticated 
industrial problems. Level III provides the student an opportunity 
for individual study and an in-depth problem solving experience. 

He is encouraged to select a suitable problem and investigate it 
in terms of the conceptual structure of industry. 

All three levels are' viewed as transitional subjects between 
general and vocational education. Their goal is to provide 
experiences that assist the individual to make wise vocational 
choices, to understand his chosen role in our complex industrial 
society, and to be a productive member of society. Only the first 
level is viewed as a required course. After this point a student 
may decide to continue in the American Industry sequence or elect 
to take any of the other programs available in his school. 
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A more detailed description of the American Industry Projects 
rationale may be found in 'The Establishment of American Industry as a 
Transitional Subject Between General and Vocational Education*' authored 
by Wesley Face and Eugene Flug, Co-Directors of the American Industry 
Project. * •' 



J <w 



t 



I 




t 'y,) V X-v; 'j.' > . 2* V;*f^ V 

"rft.**.. 4MX f * ’*r tfVi-A., 

^Ss; «.3/V -■•• ■• ilr Vivij 



Kmw. W:-ir 

A # f V i ru* iKV 2 • ^ «•. 7 



PROCESSES 



r* v * /> v #■ f.^r ^ 



f lots 

wS * \ I Hamj 4 • 

iHi ? * v, *«J. \ 



V$5k> •■> ••< •*,**<& 





# I 



Figure 1 

A CONCEPTUAL STRUCTURE 
OF THE KNOWLEDGES NECESSARY 
TO UNDERSTAND AMERICAN INDUSTRY 
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Evaluation of -the American Industry Cou rses 



Purposes of the Evaluation 

• • • . • 

Will students have significantly different competencies after 
, studying this course? Can eighth grade students read this material? 

Should. this lesson contain more detail? Why was 'this activity success- 
ful while the first one failed? These and similar questions challenge 
curriculum, ciovclopers and evaluators. And; the American Industry Project 
was not unique in this respect in that it had to answer the same questions. 

The first question may bo answered by measures of terminal behaviors 
in experimental* quasi- experimental; or time- series experimental studies. 
However,. the other questions listed require additional,, information. Most 
of this information is related to the inputs into the instructional ' * 
setting and the teaching-learning activities that transpire in this same 
sotting. In many ways this information is analogous to the data required 
to malcc scientific management decisions. 

As the Project bogan the research specialist identified several 
questions germane to the evaluation. Host of those wore associated with 
the outcomes of the study of American Industry such as the ability to 
think conceptually and solve problems.. However, after the data for the 
first term had been analyzed, a mixture of positive and negative results 
caused a further examination of the scope of the evaluation. Additional 
questions which emphasized the gathering of data pertaining to the factors 
that influenced the terminal behaviors of students woro defined. Then 
the evaluation system was revised to provide the means to collect the 
information required to answer those questions. Not only did those 
additional data assist in illuminating the critical factors related to 
the end results but they helped to identify problems in time to take 
corrective action prior to the end of the school term. 

.The. complete evaluation system presently employed by the Project 
t ^scribed in the next section. Ro, member that it was developed to 
assess the outcomes of the study of a new curriculum and to provide 
management information to the Project staff. Each instrument and data 
gathering device was selected and designed to provide the information 
•roquired for. decision making and .answering the critical questions 
Associated with the development and teaching of those courses. 

• I* is impossible to overemphasize the importance of distilling the 
critical questions and identifying the, essential decisions as the basis 
'for the design of an evaluation system. Evon if it proves difficult to . 
obtain information for some of them, this very fact will stimulate the 

researcher to further development and encourage creative ways to garner* th<5 
needed data. 



Evaluation Domains and Procedures 



Early in the design of the evaluation system three domains 
were defined in which to collect data. The ingredients domain 
encompasses all of the inputs into the learning situation. Examples 
of these inputs are the quality of the instructional materials, 
abilities and interests of the students, characteristics of the 
participating teachers, and the' intellectual climate of the school 
and the community. 

As a course is being taught, a number of instructional 
processes are applied to and interact with these inputs. Students 
are exposed to instructional media, activity sheets, and booklets 
in the American Industry course. Teachers are exposed to new 
materials and ideas and their associates become aware of a new 
curriculum. All of these on-going actions and activities are 
embraced in the processes domain. This domain can generate a 
continuous flow of vital information during the time the course is 
in progress. However, it requires systematic monitoring. and a 
relatively rapid feedback process. 

The third domain consists of the products or outcomes of the 
study of American Industry, it is concerned with the nature of 
the student at the end of the course, impressions of the teachers, 
reactions by administrators and staff members at the school, and 
opinions of the parents. 

Figure 2 gives, a graphic presentation of these domains and 
illustrates the relationship between them. Examples of data 
collection procedures and instruments 1 for each domain are presented 
in the next section. 

The evaluation domains identify the sources of information in 
the '’evaluation. Equally important are the types of data collected 
"and the collection schedule. Figure 3 depicts the strategy utilized 
in developing and evaluating learning expedients.. The ball at the 
"left symbolizes tho basic role tho Project's rationale plays in 
each action and decision. . Initial tryouts of materials, learning 
activities, or instructional methods probably will not provide 
optimum solutions) however, they will provide feedback for further 
refinements . These results and relationships are indicated by 
the spiral line around the solution axis and the decreasing 
distance between tho two Tilth each succeeding spiral. This 
strategy may be applied to an entire course, a unit in the course, 
or a sub-part of the unit. Its .application will depend on the 
data required by the researcher. 

The relationships of the linear distance traversed along 
the solution axis during each succeeding spiral is donoted by 
X>Y>Z in Figure 3. In other words, progress becomes more diffi- 
cult with each successive spiral. The first trial identifies 
obvious areas in which improvements can be made. As these changes 
are made, smaller and more subtle problems come to the forefronts 
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Each spiral along the solution axis contains five distinct segments 
that represent specific steps in the development and. evaluation of a 
curriculum or a sub-part of it. The first step is to state the ^actives 
for the course or segment of instruction in as lucid Ucjrms au poosibio. 

Once the objectives are stated learning expedients can be solecteo. or 
developed to meet thorn. At this point the learning expedients arc reaoy 
for a tryout in an instructional situation. This nay be m a repilar 
■ classroom or a special group of students may be selected on ™hic 1 o 
the first trials. Data are collected prior to, curing, and at tlx. ena 
of the tryout to provide the basis for tho evaluation. Tno ciomains noted 
previously identify areas in which to collect information. Interpretation 
of the products data is made in relation to the ingredients and processes 
data as well as in comparison with the performance of the control g 1 • 

A graphic illustration of a specific spiral is given in Figure U«T o 
ring which envelopes the spiral connotes the role 01 the ingredicnuSc. 
processes data in evaluating the outcomes of tne oest of tho learning 

expedients. Q^pi^tio^ ^ ^ evaluation, tho researcher must decide 

whether tho objectives wore met and if not, tho ruason(s) for not att 

ing them. If the decision is made to continue into another 

same five stops would be employed. This process would bo continued until 

a satisfactory level of achievement is reached. -'Satisfactory as used 

in this context does not imply that tho instructional materials arc. 

perfect, rather, it means that in relation to tho other elements 

instruction being developed the one under consideration . has 

to tho point where it would be more efficient to work wit a another one. 

To the degree time and resources permit, a variety of measurements 
should bo made during each evaluation. Each type ox measurement has its 
biases. Use of several will hopefully isolate too lactual information 
from tho random error and bias factors present in specific typos of 
measurement. For instance, objective tests allow *ohe evaluatortosamplo 
a proscribed set of behaviors but they penalize tnc nonverbal stuaent. 

Whereas essay tests allow tho student to freely express iiimsclf, they 

suffer from the lack of scope in the behaviors . sampled and tho 

of tho scoring process. Observation of students In une normal class ro 1. 

setting and engaging them in discussion can also nolp to a3S0S3 t ivi 

competencies, but this procedure in itself alters one normal 

situation and is open to rater bias. However, use ox the data from all throe 

sources in tho analysis can help to identify tho crue outcom . 

Whan using this approach car; must be taken to avoic overloading any 
one person with evaluation forms. Students could, be inundated with tests 
and rating sheets. Since there are a variety of sources of informations 
this situation usually can bo avoided by some careful planning. The 
1 Amorioan Industry Project, for example, obtains information- from tno . 
participating teachers, students, guidance counselors, administrators, 
the Project’s supervisor of participating teachers, and too instructional 
materials. Tfaero possible, simple forms are designed to facilitate 
‘ responding a rd direct duplication of da La is avoided with tno exception of 
some collected to use in determining the reliability of the measuring 
techniques . . 
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Figure 4 

A Specific Research and Development Spiral 





- 10 - 



The following arc some examples of the data contributed to the Project 
and their source. Individual lessons and other instructional materials • » 

are rated by the participating teacher. Rating sheets are placed after 
each lesson in the instructor's guides and accompany the instructional • 
media. These forms require two to three minutes to complete and provide 
useful information on the structure* contents* and utility of the material 
rated. Also* the placement of the rating sheets provides systematic 
feedback which is important in the evaluation. The time intervals between 
feedback provide an indication of the progress of the course. Workshops 
and the end of the year report also provide the teachers with opportunities 
to communicate their questions* problems* and opinions of the courses. 

Students express their opinions of American industry courses on two 
forms. A free response opinionnaire . is completed at the end of the course. 

In this form* the student lists the classes he likes and dislikes and the, 
reason(s) for h 5 .s listing. In addition* he is asked to list his likes and 
dislikes related to the American Industry course. Several statements in 
the rating form employed to evaluate the written materials focus on the 
students * opinions of the quality and utility of the instructional materials . 

The students 1 knowledge of industry and problem solving skills are assessed 
by means of the Project's achievement test. This tost has Objective and 
essay portions. Their interests and activities are recorded on the Project's 
Student Questionnaire which is comprised of items selected from Project 
TALENT'S Student Information Blank (I96I4.). 

Guidance counselors provide ability and reading test scores to the 
Project. Administrators complete a. form which describes the resources available 
in their schools and the curriculum offerings. They also comment on their 
•school's relationship with the Project and raise any questions they may have. 

The supervisor of participating .teachers attempts to visit each partici- 
*pating teacher once 1 a month while school is in session. During these visits > 
he records the progress and problems identified by the participating • teacher « 

Ho also notes the activities of the classes and interviews individual - 
students . 

A more subtle but very appropriate source of information is the condi- 
tion of the instructional material when it is returned to the Project. 

Materials that have been used will show signs of wear* pencil or pen marks, 
and smudges. For example* several of the Project's overhead transparencies 
have a place to write in comments as the class discussion progresses. A 
•clean transparency and mask indicate .that it was not used in the manner it 
was ^designed for. Booklets with. clean covers and tight bindings suggest 
that .they were not opened very frequently. . . 

The data collected are channeled to the research specialist for 
analysis and interpretation. During this stage of the evaluation, several 
comparisons are completed. Comparisons are made between the students in 
the American Industry classes and the students in the control groups. 

In as many schools as feasible* experimental designs are used for the 
evaluation. However* in seme situations this is not possible and quasi- 
exporimcntal designs have to be substituted. 
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Data from American Industry classes during previous school terns 
are aSo used in the analysis to identify any improvement or deteriora- 
tion in perfomancc over the given time period. These changes can 
then be correlated with revisions made in the courses, instruction 

“"^Tfe'kSct^ofSduftsIf' the study of American Industry can 
also be associated with the appropriate ingredients anc processes. 

«n nva.mnle of the interrelationship between tno Jiroe domains is , 
provided^by°the evaluation of tho student booklets used in American 
industry classes'. The following data are colloccca: 



Ing redients 

Reading level of 
each booklet (Dal< 
Chall formula) 



Reading ability of 
the students 
(Reading test) 



General ability of 
the students 



Pr ocesses 

Teacher descrip- 
tions of uses made 
of the booklets 
(Feedback form) 

Supervisor’s of 
Participating 
Teachers observa- 
tions of the class- 
■ room use of the 
booklets 

Signs of wear and 
usage on the book- 
lets 



Products 

Student opinions 
on reading level 
and content 
(Reading form) 

Teacher opinions 
of the utility 
of the booklets 
(Feedback form) 



Student • . 
achievement 



If student achievement is low in an area t nor ougnly .discussed in 
tho b k t tho possible cause or causes can bo .defined and evaluated. 
For instance* tho students may not bo "able to reac, the material. This 
cal' be assessed by comparing the .reading level of the material, the 
•students * reading abilities, and thoir opinion 01 the readability, of 
S materiS! in like manner, other causes of variations in performance 

my ^Host of this information is very important to tho curriculum 
specialist? 1 In some instances it will verify his decisions-- inothcrs 
iWll point out now directions or problem areas which noon attention. 
Henc- it is important that the researcher work closely with the curri- 
cSui* developer and that they comprehend each other s problems. On the 
American Industry Project this has been accomplished with substantial 
benefits to both functions. 
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Selected" Evaluation Procedures,. Instruments, and ‘Outcomes 



Previous pages have described the general nature and purpose of the 
evaluation system developed for the American Industry Project. Some^of 
the evaluation procedures, data gathering instruments* and resulwS^a^ 
now be presented to illustrate how the system works and the type oi. out- 
puts generated by it. This is not a complete enumeration, rather, it is 
•a sample to demonstrate" the' characteristics of the system and the typos 
of data collected: 1 *A full report on . the research carried out by the 
Project will not bo available until late in 196$?. 



Organization of the Instructors Guide lor Level I 

One of 1 the' first problems encountered by the Curriculum specialist 
was the design of a functional instructor’s guide. This guide hao to be 
flexible enough to be employed in a variety, of industrial arts labora- 
, tories by industrial arts 'teachers with diverse backgrounds . At the same 
’ time, it had to' 'embody the American - Industry Project’s rationale and. take 

into account the nature of the learning process. 

Three separate aspects were involved. in designing tne guide:, ^1) design 
of the format/ ('2) selection of the content, and (3). specification of the 
sequence of material. All three of these elements are discussed. in detail 
•? rin.a paper by Richard Cxebhart (1963). Thus, --this paper will review e ?. . 

'the degree necessary to make the data gathering procedures and the resulting 

data meaningful. , . ^ „ ■ 

The lesson format evolved-' over a three year period of experimentation . 

with various forms. In the present form, each lesson consists of objectives, 
scone of the lesson, references, and content outline. The content portion j 
pf--.,the lesson is written in narrative form.. Material that is to be dis- \ 
cussed, or covered in discovery mode is set off ira parentheses. Lessons \ 

vary in detail depending upon' the teachers backgrounds in the. area involved. . 
For example, the marketing lessons were writtc n m.a very detailed fashion 
while the processes lessons contained concise outlines. Anotner feature Oj. 
the content portion of the lesson was the provision for utilizing instruc- ; 
tional media. Instructions for using, the narration, and the discussiori i 
questions x*ere placed at the point in the lesson where the instructional ; 

media was to be used. _ , .. ' 

Evidence on the utility and effectiveness of this lesson format and the 

design of the lesson content is presented in Table 1 . The two items 
directly related to the format of th o le ssons in the guide, numbers seven- 
teen and thirty-nine indicate that the fornar^^^ional and. easy to 
follow. There is evidence that the teachers had some proo^ems interpreting j 

the parentheses (item twenty- two ) . I 

H. * 



O ' 
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Tablc 1 

* *• • '■ '■'Res^brisGs ' on tKef 1967-63 IM of Year Report . 

* Related to Lesson Format and Content. . • 

.Item jjbft . Statement • Bps^is qs 

• . .. ‘ *’* ' SD D U . A SA NR 

‘ 8 ;. !. . jr. could not understand tne, material in • . ’ 

• the Instructor’s Guide 9. 3 U • 

17 . The format of the guide is easy to 1 10 6 

follow. • ••••. •.,*••• *. : ... ..... 

.. 21/ ■ •' .. The narrative in the lessons did not give • • . 

. enough, content. .« /••.•* .4 . 5 . 

„ • * . • . 

,, «i •* 4 *• . # •* • * *•*, * . / * I 

V’ 2 ' 2 .. • I. could tell by the narrative in the 

• . .lessons when i was to use discussion • ' o o 

or other student involvement - . . ; ' . , 

33 . .• . .. The lessons did not go into enough . • , 31 

: i‘ 1 r .. •.• •• depth.'. •%. j 

— 3U- • • • .The instructional media - were. effectively . 

correlated with the lessons ' 

, • , • • The format of the lessons is functional. • ^ ^ 

'*• ' . .' . • . • .. . •• - • 

. Key to ' Responses: * . SD * Strongly, Disagree, D - Disagree, U ~ 

V; . . ’. a =; Agree, SA. Strongly Agree, NR = Wo Response < 

# ‘t . • # . 1 • 

».i t • *. .* • * # * 



& 



si* 



r • 

, * , 
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\ * 
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^Xtera numbor in bho End of Year Report. 
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• Host of the teachers felt that the lessons wont into 'sufficient 
depth and provided enough information (items twenty- one and thirty- 
three) . However, there wore enough uncertain and agree responses on 
item twenty- one to stimulate further investigation of specific weak- 
nesses. As a result, additional instructional materials., were mailed to 
the teachers and workshop presentations were selected to provide more 
content background for the participating teachers. About seventy 
per cent of the teachers responded that they could understand the 
material in the guide (item eight). The actions mentioned above should 
also help the thirty per cent who had problems comprehending some or 



all of the guide. 

Over eighty per cent of the participating teachers had little o 
no trouble integrating the instructional media with the lessons during 
their instruction (item thirty-f our) . This information substantiated the 
decision to include the scripts and directions for the utilization of 
the media at the point in the instructional sequence at which they were 

to be used. , _ ' . - _ 

• • • A more difficult and intriguing problem was posed .by the question 

of the best sequence in which to study the concepts and structure of 
industry. Psychological research suggested that a cyclical presentation 
of the subject matter would be most efficient. Through encounter with 
learning experiences related to a concept in a variety of situations and 
contexts the student has & chance to periodically refresh his memory 
and embellish this concept of the material being studied. To illustrate 
this point, consider the cyclical study of the concept of communication. 

In his first contact with the materials and activities related to this 
concept, the student will have only a slight knowledge of the other . 
concepts in the structure of industry. Hence, the concept of communica- ' p 
tion formed by this student will be relatively simple and will be connected 
to only a few industrial referents. However, as the study of American 
Industry progresses throughout the school year, communication in its 
various forms will be revisited on several occasions. By means of this 
sequence, the student will experience communication activities in such 
areas as planning, production, management, procurement, and marketing. 

Thus instead of forming a unidimensional concept, the student s concept 
of communication will have n dimensions where n should at least approach 

twelve and may exceed this number. , . . , 

Two techniques developed by Gagne* (1965) were employed to determine 
the information and sequence of information to present in each cycle. 

Starting with the problem solving, and knowledge of industry objectives 
established for the first course, the curriculum specialist and the 
research specialist determined the knowledges, concepts, and skills required 
to attain those objectives. The analysis proceeded from the two broad objec- 
tives back to the characteristics of the incoming students. Care was taken 
to identify logical and complete sequences of learning. Figure 5 presents 
a very brief summary of the tnxonomical breakdown that resulted. 
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aftar this analysis hod been completed, the course outline was 
established in S finalf orm . (Figure 6 gives the Level I course outline 
and identifies the cycles in red.) Lesson themes wore delineated -nd specific 
dcills and information wore allocated to the lessons on the basis of the 
analysis discussed in the previous paragraph. Tho information in the 
lessons included the appropriate interrelationships among tho concepts. 
Iheseinterrolationships a?o vital to tho study of the complete structure 

“ -sa. sfrsrss siart «.-«» 

m4or to vrttir... thoro still • qu.stlon as to tho otrootoro and 
content of the output- With several writers' and pauses to complete o 
jobs there was a distinct possibility that some olomonts had been 
or placed in improper perspective. To evaluate this P°ssii y> 
analysis procedure was developed by the research specialist. 

An analysis matrix was dovoloped with numerical labels for each 

concent, the attributes of the definition of indus ry, an analysis'" sh«>et 

on the horizontal axis. The vortical axis in part one oi the analysis sh-et 

listed the major portions of the lesson. In the second part, which was 
concerned 6 wdth°the interrelationships established with other concepts, the 
vertical aids had the same categories as tho horizontal ax . 

Table 2 presents tho concepts, attributes, basic terms and the 

numerical labels used in tho analysis. Output f yS p? °^es^ Within 

first communication lesson in Unit III is given in Ta ^° 3 ; Figures witfli 
the matrix represent levels in tho taxonomy for tho cognitive domain 
develop^ bv Bloom. The left-hand column identifies in order the low 
tSonow level present in the introduction, the high taxonomy level present, , 
tto Ser of instances at each taxonomy level, any introductory activity 

utilized, and the industrial problems included. The s f'° 1 “?? e 1 ana 1 ysis . 

utilized for tho body and sur.mary portions of the first part of the analysis. 
A further explanation would probably be hclpful in oonvoying the •_ 

meaning of Hnatanoes 1 ' as employed id. thin the context of this analysis. By , 
meaning oi _n. i. in3truct ion vdiioh can stand by itself. 

Consider for example, a "specific illustration of communication in industry . 

’ ducin- too fow parts because of faulty instructions. This ; 

nrovides^an "instance" to discuss in introducing tho study of communication. 
Kdn as th"s directs its attention to another illustration or topic,.. 

“ “Vrt" ttooS^able 3, discloses that the introduction 

rasKr'TAS.r*-. i?s** a 

VSiiVSZ 2i fSSst - 

students ar^ ranuired to interpret and translate material related to 
instruction. Also, the attributes of communication aro mentioned (note the 

1 ^ Knally^i^th^summary tho students review tho material and tho 
teacher test3 their concept of communication by asking thorn 
a unique situation. This 'tost" is indicated by the three at the high 

taxonomy level* 
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LEVEL I - .AMERICAN INDUSTRY: COURSE OUTLINE 

TEACHER DIRECTED • 

Units and Features Unit Theme 



Unit 



Unit i: 



c 



Unit V 



Unit 



Unit VI] 



o 

ERIC 



Industry Today 



1. Introduction to American 
industry 

Resources of industry 
The environment of industry 
teic^xiTsr^fU^ndustry 



Let's analyze 
.industry. 



TP] 



The Evolution of Industry 



]he needs of man 
search for greater 
'oductivity 



The needs of 
lan and why he 
progressed. 



Some effects of man's 



XC Organizing an Enterprise 



start a 
busing 



1« Communication 
2. Research 
3* Management 
ll» Finance 
5>* Property 
6. Energy 



Unit IV Operating an Enterprise 



1. Relationships 
2 • Procurement 
3* Materials 
2l» Processes 
5* Production 



Let's produce 
using modern 
production 
methods • 



Distributing Produces and 
Services 



Why does 
duct epH? 



pro- 



1. Marketing 

ianapor fcstioir 



The Future of Industry 



lere do we go 
»om here? 



1* State of the art 
,2. Reasons for change 
3 • Future expectations 



The Students ' Business Venturer s, The students 

arganize, pro~ 



1. Development 
2 • Fabrication 
3 * Marketing 
h Evaluation 



dlice, and sell. 



Figure 6 



STUDENT ttri 
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Table 2 . 

NUMERICAL LABELS FOR THE AMERICAN INDUSTRY 
STRUCTURE AND DEFINITION 



A. Ball Concepts 



C 



Numerical Label 


Concept 


1 


, . Communication 


2 


, . Transportation 


it 

5 


, . Finance 


. . Property 


6 


. . Research 


7 


. . Procurement 


8 


. . Relationships 


9 


. . Marketing 


10 


. . Management 


11 


. . Production 


12 


. . Materials 


13 


. . Processes 


Ik 


. . Energy 


Ring Concepts 


Numerical Label 


Concept 


20 


. , Government 


21 


. . Private Property 


22 


. . Resources 


23 


. . Competition 


2h 


. . Public Interest 
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C. Attributes of the definition of industry 4 . 

Numerical Label Concept 

||0 institution in our society (also indicates 

the entire definition of industry) 

ill Applies knowledge 

1*2 .... .. Utilizes resources (human and natural) . 

to . . Produces goods and sorvicos 

kh . . . . Meets needs of man 

1(5 .... For monetary profit 



D. Other basic terms usodj 



Numerical Label Concept 



Jo Attributes, characteristics, or similarities 

.• 51 ...... Concept, conceptual 

60 Productivity 

70 Student booklets and/or reading materials 



t> 



£ 



o , 
ERJ^C 



- 19 - 



HMCO^tnOMJOO'O^N 

OOOOOOOOO^ 

oooooooooooo 

oooooooooooo 

fOfAfA^mmfOfAfon^mfO 

o o o o o o o 



CO tf> 

H 

o o o 

rM »**1 rH 

o o o 
rA w (0 

f**l f - l 

o o o 



gjN-COO 
H H H 

o o o o 

HH 

o o o o 

ro ro m f a 

h( **• 

O O'O 



OMfMfA'tf'iAOf-COO'OM 

rj(\jojtNrvj{M<Mr-JC\jcMrom 

oooooooooooo 

,u| ^4 r*» H rl rl H I~l W 

o O O 0.0 o o o o O O O 

ro fo fn co <A ro ro 

hhhhhhhhH ^ 



o 

V 

(A 



(M 



(14 

H 

< 

o 



z 

o 

to 

to 

U4 

•J 



o o o o o o o 
«0 

M O o o o O O 

(A 

o o o o o o o 
(A _ 

tn o o o o o o 
**■ 

lOsfOOOOOO 
LU < 

►-MOOOOOO 
O sj* 

03NOOOOOO 

ShOOOOOO 
h *1' 

|w O *** O H o o o 
■< <• 

■ # o o o o o o 

N 

cn o o o c> o o 
cm 

NOOOOOO 

04 

o^oooooo 

WOOOOOOO 
C£ AJ 

m* o o o o o o 



<0 



z 
I— o 



2 f- 
D < 

o 



«n o o o o o o 

NOOOOOO 

m c o a o © o 

H 

o o o o o o o 

H 

.JOOOOOOO 



z 
•m x> 
• se 
«j a: 

UJ O 

> o 

UJ 

«J o 



<0000000 

60 

o o o o © © 



z 

• o 

06 M 
O H- 
tLOO 

so 
V) o m 
moo; 
to OJ UJ 

>■ H o. 
«4 Z 

< ««• o 
z M 

< o 



#*0000000 

to 

HiAOOOOOO 

U4<000000 

u • 

ZNOOOOOO 

o 

ommo^oo© 



Z •# 

O 114 « 
(0 SC UJ 
V) 111 X 
(14 X M 
J h h 



o o o 
o o o 

HOH 

o o o 
o o o 
o o o 

o O-iO 

o o o 

o o o 

o o o 

o o o 

o o o 

o o o 

o o o 

o o o 

o o o 

© © o 

o o o 

o o o 

o a o 

o o o 

o o o 

o o o 

o o o 

o o o 
• 

o o o 

M CM ffl 



oo o 
o o o 
o o o 
o o O 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o O f 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
o o o 
O o o 
o O o 
o o o 
o o o 

ro O O 



> > 

IU (U 

J jQw>(fl 
JlhO 

kx ** ec 
< < v~ > a. 

H < t* 4 

|*« q 

• o m o a o z 
Z -4 X 2 2 < M 

o 

m o o o o o a 
ec cc cc ec cc cc 

C£ h* h* h* ►— 4- h~ 

o z z z z x z 

IX M M M H N H 



o o o o o 
o o o o o. 
a o o a o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o oo o o 
o o o o o 
o o o O O 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
o o o o o 
© o © o © 
o o o o o 
o o o o o 
o o o o o 



o o oo o 

o © o o o 
*0 

f -4 o o o o 

(A 

o o o o o 

(A 

IftOOOO 

•t 

VlsCOOOO 
UJ M* 

H r» O o O O 
S sh 

dNOOOO 

C^HOOOO 
h »$■ 

h O H O N O 

< *fr 

sl-OOOO 

Roooo 

CM 

NOOOO 

CM 

0^0000 
2» <M 

M o o o o o 

OS CM _ 

, o o o o 



r\ o o o o 

NOOOO 
.MOO op 

r-i 

o o o o o 



O' o o o o 
<00000 

CO 

f- o o o o 



• <1 o o o o 

t/1 

OOOOOX(-(AOOOO 

ooooo&:w«roooo 

ooooo<znoooo 
ac o 

*M CO M cm O O o O O o 

to 



> > 

UJ UJ 

J JO>^ 5-0 
JIHO 

xx mo: 
< < 4— V* 5 s * 0* 
hH<<r r . 

K* Q 

o m O O 0 2 
J x 2 2 < *-* 



O Q Q O O O 

o o o o o o 

CQ CO C 3 CO CO CO 



>• X 

XX H (A 

< < M Z 

hhOM>0 

•J X M M 

o M 4 - H 

< < < toJ 
>** >* (44 

ec o: o o 2 o: 

< < z 2 o ec 
as x </» uj 
a: s: s: x <✓> ►- 

DDDDWZ 
to to tO to -J *-* 



-1 X 

CO UJ -4 < X 
h > IU h ^ 
ec uu > 

4—.J UJ 35 

«4 (J ^ 

< x ox 

< X 

CO M <E ►— h* 
V- h* < < 

Cl 2C 

IUOhOO 
0 0X22 
2 

O M M H M 

o o c o o 



' » 






CA 



EM 



1 * 

hLJt 






Lesson Analysis Output 




A look at tha interrelationships matrix in the bottom portion of . 
Table 3 reveals that the concept of communication was related to 
industry. ^During Unit I the students .acquired a concept of industry, 
now, in Unit III they start to study'the major concepts encompassed 
•in the Project's structure of industry. Hence^ this interrelationship 
is logical ancl is in proper sequence to demonstrate the role of communi- 
cation in industry. * . 

All of- the lessons in the Level. I instructor's guide; have been 
analyzed with thi3 procedure. In addition to the output for each 
lesson, the data cards can be sorted to. provide a listing, of each 
•lesson in which a concept appears. ' Placing these In the sequence in 
which the. ; lessons appear in the guide reveals the sequence and 
distribution of the instructional’ materials related to each concept. 

These can then be checked against the original design of the guide.- 
For instance, on a previous page it was noted that the guide would be 
written to present materials related to communication at several points 
within the course. One example given was the role of communication in 
advertising, thu3, one should find this noted. in the analysis for one 
or more of the ^marketing lessons. Indeed, the analysis for lesson four 
in Unit IV indicates that the marketing concept, number 9, is interrelated 
to the communications concept, number 1, as shown in* the portion of the 
interrelationships matrix reproduced below. 



. * • • < . i *. , • 

INTERRELATIONSHIPS MATRIX ' .:•••. •• ' 

CONCEPTS. BALL RING ATTRIBUTES ' 

CONCEPTS ATTRIB 1 2 A 5 6 7 0 9101U213142021222324404 1424344455051601 
09 LOVi TAX LEVEL 102020000020000000000110000 
09 HI TAX LEVEL 5 0 0 0 0..0 00000000000000004000 0 
09 NO AT LOW TAX 10 10 1 0 0 .0 0 0 1 0 0 VO 0 0 0 0 0 0 1 2 0 0 0 0 
09 NO AT HI TAX .1000 0000 0 0000000000000 1 0000 

* • • • * .• V i. . 



Interpretation^ of the numbers in this matrix indicates that students 
study the relationship of communication to advertising starting at the 
first level in .the taxonomy and progress to the synthesis level (level 
five) • Ttiis synthesis is attained through the actual design and production 
of advertising. * 

The data from this analysis have been helpful.’ in making minor revi- 
sions in the guide during the pact year. From this experience, it was 
also evident that this information will be very useful when a more complete 
study and revision's of the guide are made. 
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Up to this point the techniques for structuring instructional 
materials and reviewing the characteristics of the completed 
materials have been described} however, one of the most critical 
tests remains — the teachers* evaluation of their effectiveness in 
the classroom. To acquire this information, feedback sheets were, 
placed at the end of each lesson. Item eight in this form - asked the 
teachers to indicate if the lesson rated, fit into the overall 
instructional sequence. There were 571 responses to this item witn 
iki of these responses indicating that the lesson rated fit into the 
instructional sequence. Sight responses evaluated a lesson as not 
being in proper sequence. And, on sixteen sheets this item was 
left blank. Item five in the feedback form provides- information 
on the structure of the lessons. Only eleven of the $6l\. responses 
to this item said that the lesson was not logically structured. Thus, 
the data provide conclusive evidence of the logical structure and 
sequence built into the instructional materials. 

Whereas the instructional materials discussed in this section 
had to be constructed on the basis of techniques that had not had 
extensive use Or formal ovaD.uation, the data collected during 
the development and tryout phases provided an essential source of 
information for evaluating the quality of these materials. In 
addition, these data can also be used to assess the validity of the 
procedures employed in developing the instructional materials. 

The operating characteristics of the data collection procedures 
provide evidence on their efficiency and utility. Thus, relevant 
information exists for further revisions of the materials, the 
procedures used in constructing them are substantiated by empirical 
evidence, and the characteristics of the data gathering system are 
known. 



Products of the Study of American Industry— Soudent Outcomes 

Two aspects of the evaluation of student outcomes will be reviewed 
in the ensuing pages. First, a brief description of the Project's 
achievement teat and a general overview of the performance of the 
experimental and control groups on the test will be given. Second, 
selected attitudes, interests, and opinions of a sample of the 
students involved in the evaluation will be summarized and discussed. 

Since the American Industry courses were based on a unique set 
of concepts and structure of the knowledges of industry, an achieve- 
ment test had to be constructed to measure these constructs and 
content. As the specifications for the test were distilled the 
decision was made to limit the time required to complete this 
instrument to approximately fifty minutes. This was done to reduce 
the amount of time the students spent completing evaluation forms 
and to deter an overemphasis on one evaluation technique. If the 
group of students iri the evaluation had been larger , tests could 
have been made for each concept area and administered to sub- samples 

of the group. 



The basic task in writing the test was one of devising items that 
would efficiently sample behaviors critical to tho objectives of the 
American Industry courses. Since the essential aspects of these courses 

were the study of concepts* the. solving of problem's* and the study of_ 

structure* the majority of the items in the test. wore related to one 'or 
more of these elements. In addition, some Items at the knowledge "level 
were included to assess tho validity of the taxonomical structure of the 
levels of learning posited in Figure 

Items were written for each conceptual area and problems were 
defined which would cut across several conceptual areas in order to test 
the knowledge of the interrelationships of the concepts. . In addition* 
several items wore written which could be answered on tho basis of ’’pure " 
conceptual thinking without recourse to the concepts of industry. 

Appr oxima t e ly eighty per cent of the items in the final form of the test 
were- concerned with. -applying the concepts and structure of- industry to 
problems or situations in industry. Five per Trent testbd -conceptual 
thinking and the remaining fifteen per cent assessed factual knowledge. 

The .items in the objective portion of the test are four choice 
multiple- choice’ questions. Inmost instanccs-a problem- situation 
provides the basis for the item stems. When possible the alternative 
responses for. an item were constructed to provide information on the 
responder's cognitive structure or lack of -knowledge . • -An illustration 
of this is given by tho test item reproduced in Table h» In this 
problem.* .. the... dis tractors disclose whether the student has information 
on the cost* insulative qualities* and strength’ of.- common. materials. • 
If* for example* a student thought that materials were selected for 
their strength* he would select D as the appropriate choice. 

The ..response, distributions for all of the American Industry and 



control group students 


on this item" are'? ~~ — 


s ....... 


Response 


American Industry 


Control 


A 


. 7.32 . 


11.1*2 


B 


71*. 12 


57.02 


C 


13.92 


21.22 


D 


U -72 


10.1*2 



i 

These data indicate that the-, control group students are more likely to 
base their choices on the more saiient characteristics of materials. 

Comparisons can also be made between a control group and the 
classes taught by a specifi.c teacher to determine if a particular bias 
is j.nherent in his instruction. For example* last year it was found 
that the students in one school were prone to select the distractor 
. based on the finance concept when it was included in an item. This 
fact was communicated back to the teacher. In addition* this informa- 
tion suggested a need for problem solving exercises to develop the 
students' ability to deal with several relevant factors when solving a 
problem. 
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Table i| 

CATEGORIZATION FORM 'FOR EVALUATION ITEMS • 



Directions : Fields are defined by capital letters., card columns by numbers 

reproduced on the master sheet. Numbers or signs written on 
this form are to be punched in the columns designated under the 
blank. ■ • 

A. Item number; b 0 1* f> 0 

uni 

.Concepts evaluated: B. First order: b 1 Z. 0 0 0 

• * ‘ - “3“7“5~9 10 IT 

C# Second order: b_ JD 1. 0 _0 _0 D. Third order: b __0 _j3._0 __0 

12 13 15 15 16 17 15 19 20 21 22 23 

E. Fourth order: b 0 k • 0- 0 6 F. Relative Emphasis b 0 1 

25 2"f> 23 27 2 8 29 on concept order: 30 31 32 

G. Taxonomy level: b 5. 0 0 0 H. Type of item b 0 1 

33 35 35 35 37 35 39 5o 

I, Difficulty: b 6 6 (#) J. Discrimination: b • 5 3 6 1 

51525355 55 55 57 55 59 



Author of. item: 



Why would mahogany' paneling be used in 
the office of the president of an 
enterprise? 

A. To reduce building costs 
#B. To communicate his' status 

C. To improve the insulation of the office 

D. To strengthen the office walls 







Each test item was written on the. form presented in Figure. 1;. . 

The information recorded in the first portion of the form identifies 
the concept or concepts related to the question, in this example 

• the concepts are materials' (12), communication (1), relationships (8), 
and finance ( k) * Each of these would have an equal bearing on the 

■ solution; thus, the relative emphasis is rated as equal (X) • 

This item requires analysis of the situation given in the stem. 
Therefore, it is placed at level four in the cognitive taxonomy. 
Naturally, this assumes that the students have not seen this problem 
before. To insure this, the Project maintains the achievement test as 
* -a secure test and does not distribute it with the instructional 

materials or release it for review •• Item type is listed as multiple- 
choice (1)* The numbers representing the various types of items are 
arbitrarily assigned labels. • 

Diffic ul ty and discrimination values accrued as the item was 
administered to various groups. Whenever possible, test items were 

* tried out on a pilot group prior to their use in the formal evaluation* 
After an item was placed in the achievement test, information ; oh its 
operating characteristics was obtained from the Generalized Item and 
Test Analysis Program ( GITAP) developed by Dr. Baker at the University 
of Wisconsin (1966). A sample 'of the output from this program for the 
item in Table U is reproduced below* 



ITHNO 


CHOICE 


WT 


NR 


DIFFICULTY 


R 


X50 


BETA 


36 


1 


0 


• 2 


•0.444 . 


-.1729 


to 

-9.8411 


-.1755 


36 


2 


1 


29 


• 6444 


' .5408 


-.6802 *• 


. .-6430 


36 


3 


0 


10 


" *2222 


-.5728 


-1.3305 . 


-‘.6987 


36 


4 


0 


4 


.0889 


. ' -.0300 


-14.9671 


-.0903 



Starting at the left, the first three columns identify the -item, the 
response choices, and the weight given to each response. (A. weight . 
of 1 is given to the correct response in this example.) The next two 
columns list the number of students selecting each response and the 
proportion ..represented by these numbers. Column five gives the ■ 

biserial item- criterion correlation*. ” In' this analysis, the test, score. 

is the criterion. The last two columns present the parameters for the 
item characteristics curve (Baker, 1961;). XgQ identifies the point on the 
criterion scale, distribution -of total test scores,. 3t which half of the 
people with the given criterion^ score selected the response under 
consideration. These values .are in the 'standard deviation units of the 
total test score distribution. Hence, fifty per cent of the people with 
test scores .68 standard deviation units below the mean test score 
would select "the ‘ correct choice for this item. Beta gives an approxi- 
• mate value of the scops of the item characteristics curve at X£q and 
indicates the discriminating power of the item. For this aoministration 
of the item, these parameters indicate that the item is relatively easy 
and has positive discrimination. 
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• GITAP also calculates test reliability; utilizing the Hoyt 
analysis of variance metnod. Reliabilities for the Project's 
achievement test calculated on the basis of test results from 
individual American Industry classes range from .7U to .91 * The 
median value of these reliabilities is .62. No doubt these 
reliabilities could, be improved by discarding the items which 
usually discriminate poorly. Hoiraver, the decision to keep or 
discard has not been made on this basis alone. Equally important 
was the nature of the item and the information it contributed to 
the evaluation. In other words, if there was no observable 
defect in the item and it provided important data, the item was 
kept in the test and questions were raised concerning the reasons 
for the poor results on this item. Most of these questions lead 
to a review of the ingredients and processes information related 
to the output being studied. 

Content validity of the achievement test was relatively easy 
to establish. The item form shown in Figure h clearly defines 
the concept are c ?.s and interrelations measured by each. item. And, 
the taxonomy level is identified. With this information items 
can be selected to evaluate all of the concepts and a portion of • 
the interrelationships in the Project’s structure of. Industry. 
Determination of construct and criterion-related validity will 
have to await further data collection. 

A general analysis of the test results during the. 1967-66 
school year disclosed the following outcomes for the eighteen 
and thirty-six week courses. 



Length of Course Outcomes of Comparisons; American Industry 

vs. Control Groups 



- 


A.I. 

Lower 


Same 


A.I. Better— 
Not Signifi- 
cant 


A.I. Significantly 
Better, P<.0 5 

s 


18 Week Course 




5 


3 


3 


36 Week Course 


1 


0 


3 


u 



As could be expected, the thirty-six week courses produce. proportion- 
ally more significant results than the shorter course. Likewise, the 
longer course has proportionately more comparisons that favor American 
Industry but do not attain significance (differences between test means 
in this category range from l.f? - points). In contrast, the 
eighteen week course has several comparisons in which the means of the 
experimental 'and control groups are for all practical purposes the 
same (differences in this category are less than one point). One 
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American Industry class achieved less than the control group . 

/ However, this comparison was made m a quc.si- 

experimental dasign in which the control group had higher ability . 

More detailed analyses may change the composition of ttu, 

••A.I. lower," ’'Same/' and "A.I. Better” categories. However, there 
should be little if any change in the "Significant* category. . A 
the nresent time the data collected in the evaluation are being 
placed on a disk' file for efficient retrieval 

analyses. These analyses should produce a more definitive piotucv. 
of the effects of the study of American Industry and the causes of 

theseoffecto paragraphs present and discuss responses °n a 
student opSiro completed at the end of the school te™. Some 
interesting differences appear and the data also ° 

for establishing some correlates between performance ou * . 

ment test and the learning activities encountered by the st «cents. 

At the end of the 1967-63 school year students in a sample of 
the American Industry and control classes, were asxoo t P 
the forty°statemonts listed in Tables 5 and 6 f ve/esponses were 

designated on the response sheet: -'strongly disagree, g * 

nwSided " "agree '» and "strongly agree/’ In Tables 5 and 6 these 

responses^are represented by SD,\ U. A, and SA respectively. Tne . 

sixth heading, MR, indicates the number not r 0 ®P° ada " g ;. toican 
Table ^ gives a summary of the responses made by tiio American 
industry studeSs who completed the opinionnairo. .T=hle6prosentsa 
stoilar summary for students who did not take American X^ustry. In 
bn+h tables the first figure in each column gives the total numD 
students^marking^the givfn response. The second figure in parentneses, 

li sts tho p® r do°no t^r epr es ent all of the students encompassed 

the American Industry and control groups 
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The analyses reported in subsequent paragrc.pns 

Tables 5 and 6. As noted above, these data are summed over 
several sohools^nd grade levels. In addition, it was asstanoa that th. 
level of measurement represented by the response area between. 3 and SA 
was ordinal. Hence, statistical tests wero solecuod to evaluate oroinai 

in szxsrussi. «. 

for the opinionnaire. These areas were occupational behaviors, attitude t 
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industry, interest in industry, and class activities. Each of these areas 
was selected because it would provide relevant data* The first three 
pertain to general outcomes of the course. Tha last category, class activities, 
concerns specific opinions related to the nature of industry and the. structure 
of the American Industry course. It was included to assess the effects of 
various learning conditions and activities built into the course. • 



The areas and the numbers of the items encompassed in each are listed 
below: 



Occupational 

Behaviors 



Attitude Toward 



Interest In 



Class 



Industry 



Activities 



8 


u 


1 ' 


3 


13 


7 


2 


• 11 


35 


9 


5 


12 


38 


' 10 


6 ’ 


16 


. 3 ? 


111 . . 


15 


. 17 




18 


20 


23 




19 

• 


22 ■’ 


21* 




21 


2 9 


. 26 




. 25 . 


3h 


27 




• 28 


0 


CM 

ov 


\ 

% 


‘30 




37 




31 


t 


✓ 


* 


"33 


* 






. 36 1 . 

1|0 


• « 


• 



Two statistical comparisons were made using the data that appear in 
Tables 5 anct 6 V First, -chi square values were calculated for the American 
Industry and control group data on each item. Ifext, the two groups were 
compared on the complete set of responses in each category. 



The values obtained for the chi square statistic disclosed .significant 
differences, P<.05, on ten items. Further investigation of these items 
revealed that in each case the response patterns favored the American 
Industry students. The significant items and the categories in. which they 
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• are located are listed below. 

A. Occupational behaviors 

Item 13* I will probably work in industry. . 

Item 38. There are many job opportunities in industry that 
I did not- know about before taking this class. 



B. Attitude toward industry 

Industry has little influence on me. 

« 

Industry changes too much for its products. 

Industry offers a variety of job opportunities. 

I have greater appreciation for industry after 
studying It in class.- 

Products produced by industry are usually of low 
quality. 

* 

Industry provides an opportunity for people to 
develop their abilities. 



• C. Interest in industry 

Item 6. I find it interesting to look at the designs used 
*, used in the products sold in local stores. 



D. Class activities t , 

Item >26. I had a chance in this class to develop some of 
my ideas for products. 

Of the four categories, attitudes toward industry and occupational 
behaviors apparently are influenced most strongly by the study of American 
Industry.. However, the significant items in the remaining two categories 
also shed light on the outcomes of the study of American Industry. For 
instance, the responses on item 26 -indicate that the American Industry 
students had a greater opportunity to solve problems in class. 

The second analysis entailed a comparison of the total set of responses 
within each category. For this analysis, the responses SD, D, U, A, and SA 
were given the value's 1 , 2, 3, It, and with five being assigned to the most 
favorable response. The mean response was then calculated for each group on 
each item. Comparisons were made on each item to determine which group 
responded more favorably. In addition, the response patterns were checked to 
determine what effect a change in scale values would have on. the results. In 
other words, if the extreme responses received more weight, what would 
happen to the results.? These effects were also recorded. Table 7 presents 



Item 2T. 
Item 2$. 
Item 28. 
Item 31 • 

Item 36. 

Item lj.0. 
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a summary of these data and the comparisons on individual items * 

The. exact probability of obtaining the results found in each category 
was calculated by employing the binomial expansion. Use of this statistic 
was ‘based on the null hypothesis that the responses of the American Industry 
and control groups represent the same opinions and behaviors. If this 
• •- hypothesis is true, then either group should have an equal chance. of 
responding more favorably to any given item in the opinionnaire. To 
illustrate this, consider the items in the attitude toward industry category. 
If the conditions in the null hypothesis were true, one would expect that 
each group would have about the same number of items on which its members 
responded more favorably. However, In this administration the American 
Industry group responded more favorably on twelve ,of the fifteen items. The 
probability of attaining this result if the null hypothesis were true is 
less than four chances in a thousand. 

Outcomes of two of the analyses remain the same under both scaling 
procedures* American Industry students exhibit a significantly more 
favorable attitude toward industry (P<.00h) .and response to the items 
related to their class activities (F<.033). 

No -significant differences were observed in the remaining two 
categorj.es. Neither scaling procedure would appear to affect these results. 
The occupational- behavior category contains only five Items and, thus, all 
of the comparisons would have to favor one group in order, to attain a 
significant difference. In the interest in industry category, the compari- 
sons favor the American Industry students, P’v.Op, but do not exceed the 
commonly .accepted significance level of .0#. 



An analysis of these resuits reveals several interesting outcomes* 

The chi square tests identify ten items in which there is,, a signifi- 
cant difference between the response patterns of the American Industry 
students and the control groups. All of these differences favor the 
American Industry students, a fact that is of considerable significance 
in itself. Moreover, each category in the opinionna3.re has one or. more 
items on which a significantly different response pattern was found. 

This indicates that the study American Industry had some effect in each, of 
these areas. 

Analysis of the complete set of -individual item comparisons for each 
category indicated that the American Industry students responded more 
favorably on the attitude toward Industry and class activities Items. 

Hence, it would appear that the American Industry students have had a 
positive shift in their attitudes toward industry. They also^show a^more 
positive attitude toward the opportunities and products associated with 
Industry. No doubt some of this positive effect carries over to item 13 
in which a significantly larger number of American Industry students express 
a preference for working in industry. 

The significant difference found in the set of items related to class 
activities may be of greater interest -to the Project staff than to other • 
people since these items are somewhat specific to the design of the 
American Industry courses. However, few people will probably quarrel with 



' o' 

« * 

ERIC 



■with the educational relevance of the significant difference found on 
item 26 which reads ”1 had a chance in this class to develop some of my 
ideas for products. ” These data are germane to the Project’s objective to 
develop the problem solving ability of students. The reader is encouraged 
to review the results on the remaining items in this category and judge 
their relevance for his work. 

^ / 

• 

In the area of occupational behaviors, the two items, 13 and 38, which 
were significantly different in the chi square tests indicate that the study 
of American Industry increases the students perspectives of the jobs 
available in industry and their interest in seeking employment in one of 
them. Both of these effects are important and relevant to the contemporary 
educational scene . 

As noted in the first part of this, report, the data in Tables 1 and 
2 were obtained from several schools. There was some concern that one or 
two schools might contribute most to the effects observed. However, review 
of the results in the individual schools indicated that the statistics for 
individual classes varied only slightly from those given in Tables 5, 6, 
and 7- 

Other* interesting facts of a noncomparative nature can be found in the 1 
data reported. Student interest in visiting a business, for example, was 
very high in both the American Industry and control groups. This would 
appear to be a good learning activity to satisfy student interests and 
develop new insights into industry. Also, a large number in both groups, 
about 7£ per cent, were aware of the changing nature of jobs in industry. 

The reader’s interest may delineate further items and data of interest as 
he reads the tables. 



Summary 

The evaluation system described in this paper was designed to provide 
data for evaluation and curriculum decisions. Outcomes of the study of 
American Industry form the basic criteria for making judgments and decisions 
In addition, data related to the characteristics of the ingredients and the 
learning and instructional processes are collected. This information 
provides a basis for defining the causes of the outcomes, determining 
differential effects of the study of Amen. can Industry, and isolating the 
elements that need further study. * Specific data collection procedures and 
instruments were developed to acquire the needed information. Thus, for 
example, the achievement test was designed to provide data on the essential 
elements in the Project’s rationale instead of selecting items that would 
maximize differences between the experimental -and control groups. 

Studies reported in the previous section illustrate the types of data 
collected. Also they give some evidence of the effects of the study of 
American Industry. These reports, . however, are based on a general analysis 
of the data. A complete evaluation report will be completed next year. 
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TABLE 7 



Comparison of Average Response Values 



Item 


Category 


Average Response Values 


Effects of Rescaling 
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Attitude Toward Industry 



Interest in Industry 



Class Activities 
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